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1 Introduction 

My area of research is in artificial intelligence and robotics. The major platform 
of this research is NASA's Robonaut. This humanoid robot is located at the Johnson 
Space Center. Prior to receiving this grant, I was able to spend two summers in Houston 
working with the Robonaut team, which is headed by Rob Ambrose. My work centered 
on teaching Robonaut to grasp a wrench based on data gathered as a human teleoperated 
the robot. I tried to make the procedure as general as possible so that many different 
motions could be taught using this method. 


2 Work Accomplished 

Fall 2003 

With this grant, I finally achieved this goal. Robonaut is now capable of grasping 
a wrench with fair reliability anywhere in its workspace.* The completion of this goal 
allowed me to finish my Master’s thesis [1] in December 2003. In addition to helping to 
pay for my tuition to Vanderbilt University's School of Engineering, this grant allowed 
me to visit Houston in September in order to perform the final experiment needed to 
show my findings. The following is an excerpt of a journal paper [2] that is based on my 
Master's thesis. This is extremely relevant because my PhD work is a continuation of this 
research. (The works referenced inside this quotation are listed at the end of the quote, 
not in the references section at the end of this document.) 

I. Introduction 

The paper deals with the problem of enabling a robot to learn from experience by 
building models of the dynamics of its own sensory and motor interactions with objects 
and tasks [1]. This interaction is initially provided by fine-grained teleoperator inputs. 
Over time, information gleaned from teleoperator guidance is compiled into autonomous 
behaviors so that the robot can perform tasks on its own and so that the level of discourse 
between operator and robot can become more abstract. 

... if a robot is controlled through an environment to complete a task while recording its 
[sensory-motor cooridination (SMC)] vector time-series, the result is a state-space 


* As long as the wrench is positioned vertically. The inclusion of rotation angles is proving problematical, 
mainly because it is nonsensical to add a distance in centimeters to a distance in angles. This issue is one of 
the things my current research is attempting to address. 


trajectory that is smooth during the execution of a behavior but that exhibits a comer or a 
jump during a change in behavior (an SMC event ). ... 

This paper reports the results of learning to reach toward and grasp a vertically oriented 
object at an arbitrary location within the robot’s workspace by superpositioning a set of 
SMC state space trajectories that were learned through teleoperation. The ideas behind 
the procedure are based on a number of assumptions: 

(1) When a teleoperator performs a task it is her/his SMC that is controlling the 
robot So controlled, the robot’s sensors detect its own internal states and those 
of the environment as it moves within it. Thus the robot can make its own 
associations between coincident motor actions and sensory features as it is 
teleoperated. 

(2) In repeating a task several times, a teleoperator will perform similar sequences 
of motor actions whose dynamics will depend on his/her perception of similar 
sensory events that occur in similar sequence. As a result, the robot will detect a 
similar set of SMC events during each trial. Therefore each trial can be 
partitioned into SMC episodes, demarcated by the common SMC events. 

(3) Sensory events that are salient to the task will occur in every trial; sensory 
signals that differ across trials are not significant for the task and can be ignored. 
By averaging the time-series for each episode point-wise over the trials, a 
canonical representation of the motor control sequence can be constructed. As a 
result of the averaging, true events in the sensory signals will be enhanced and 
those that are random will be suppressed. . . . 

III. Behavior Superposition 

There were four phases in the data gathering and analysis for this learning task: 

1 ) A teleoperator controlled the robot through the tasks that would serve as 
examples. Five trials at each of nine locations were performed of a reach and 
grasp of a vertically oriented object (a wrench). As die teleoperator performed 
these example motions, Robonaut’s sensory data and motor command streams 
were sampled and recorded as a vector time-series or signal. 

2) The SMC events common to all trials were found and used to partition the signal 
into episodes. The episodes were time-warped so that the yth episode in the kth 
trial had the same duration (and number of samples) as the yth episode in every 
other trial, (cf. Section III-C.) 

3) The signals were averaged over all five trials at each location to produce a 
canonical, sensory-motor data, vector time-series for each location — 

4) These generalized motions were combined using the process described by Rose 
et al. [22] called Verbs and Adverbs. 

When the process completed, the resulting set of parameters could be saved to file and 
then used to create a general representation of the task that was adaptable under real-time 
conditions. 

A. Teleoperation 

The task performed by die teleoperator was to reach forward to a wrench affixed to a 
frame, grasp die wrench, hold it briefly, release it, and withdraw the arm. The frame 
made it possible to re-position the wrench as needed while keeping it steady during task 
performance. For the purposes of these experiments, the wrench was positioned in a 
reachable, nearly vertical position. Nine example locations were chosen. Eight of these 
were positioned approximately at the comers of a virtual box that defined die limits of the 
reachable workspace. The ninth was a point near the middle of the box. Five trials were 
repeated at each of the nine locations.... 

B. Segmentation 

A vector time series . . . was recorded during each teleoperated trial of the task. The time 
series contained [over 100] separate signals ... from the various sensors and actuators. ... 



The time-series data from the experiment was manually segmented into 45 trials 
according to markers embedded in the voice channel of the robot’s data stream. Then 
each trial was partitioned into five SMC episodes (reach, grasp, hold, release, withdraw) 
demarcated by SMC events that were found through an analysis of the mean-squared 
velocity (MSV) of the joint angles. ... 

C. Time Warping: Normalization and Averaging 

Once the segmentation of die data was complete, the SMC episodes that comprise the 
task were time-warped through resampling to have a duration equal to the average 
duration of the 45 trial episodes. Then for each of the 9 locations the average vector time- 
series was computed from the five corresponding trials. For example, the reach behavior 
averaged 1 50 time steps across the 45 trials. Each of the time-series that comprised the 
reach episodes was time-warped and resampled to have length 150. The five reach 
episodes from the five trials at each location were averaged to create nine exemplar reach 
episodes each with 150 samples in duration.... 

D. Superposition using Verbs and Adverbs 

After the resampling and averaging of the sensory-motor data from the example tasks, the 
data were analyzed to characterize the motions that would enable Robonaut to reach 
toward and then grasp a vertically oriented wrench anywhere within its workspace. This 
was done with an interpolation method called Verbs and Adverbs , (VaV) developed in the 
computer graphics community by Rose et al. [22].... 

In [22] several example motions were created for articulated characters. The mapping of 
these motions into a multidimensional adverb space defined extremal points along axes of 
the space. A particular adverb extremum characterized the appearance of the associated 
motion. To create motions that exhibited combinations of the characteristics, a location in 
the adverb space was selected and mapped back into the motion space. In the work 
described here, the adverbs are the 3D Cartesian world coordinates of the object to be 
grasped (the wrench). Exemplar reach-and-grasps were acquired near workspace extrema 
for the robot’s right arm. To perform the operation at other locations in the workspace, 
the VaV algorithm was used to interpolate the exemplar motions. ... 

IV. Experimental Methods and Procedures 

The VaV procedure was tested in simulation and on Robonaut. Simulation tests were run 
on a randomized list of 269 reachable targets in a 3D grid that covered the entire 
workspace and extended somewhat beyond the edges defined by the original box. The 
test on Robonaut was performed by affixing a wrench to a jig, and placing it arbitrarily at 
reachable points in the workspace. Some attempt was made to cover the entire 
workspace, but since the goal was to prove that Robonaut could reach randomly 
generated targets, a systematic selection was not used. . . . 

V. results 

. . .The Verbs and Adverbs method ... had better than 99% accuracy in the simulator 
trials, which were designed to cover the entire workspace. While not performing perfectly 
in the physical trials, it still [successfully grasped the wrench in 20 of the 23 trials.].... 
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Spring 2004 

In the spring of 2004, having finished my Master's thesis, I started my PhD class 
work. Besides completing these classes, 1 spent the spring researching methods of data 
segmentation, as the method that I had used to segment the grasping data did not prove to 
be as general as I had hoped. I also spent some time looking into the problem of 
representing angles in such a way that the Verbs and Adverbs method would be able to 
teach Robonaut to grasp a wrench at any orientation. As yet, no reliable solution to either 
problem has been made. I'm using this year’s renewal of my grant to continue to look into 
these issues. 

Summer 2004 

In the summer of 2004, 1 went back to Houston for eight weeks. The research 
team there indicated that what they needed from me was to integrate my work with the 
work being done by several other institutions, especially UMass. Rob Platt, Andrew Fagg 
and several others (from UMass) were working on a way to let Robonaut use a power 
screwdriver to tighten lug nuts on a wheel. My work would be very helpful in the more 
freeform motions (getting the hand to the driver, and getting the grasped driver to the 
wheel, etc.) In addition to finding ways to integrate our work, I ended up taking on a level 
of my own research that I had been putting off. 

In an ideal case, the robot should be able to identify a task and make a plan of 
actions that it knows that can accomplish that task. This plan would consist of a state 
machine that issues commands to various programs, such as the Verbs and Adverbs 
program or the program created by Rob Platt that used force sensors to grasp the driver. 
The very nature of a state machine is ideal for this purpose because it has, built into it, a 
way of handling contingencies. If one of the programs it calls fails to work (for whatever 
reason), that program can report a failure to the state machine, which can take a different 
branch and perform different behaviors than it would have if the program had succeeded. 
If the state machine could not recover from the failure, it would then report failure to the 
planning mechanism, which would make a new state machine based on the new situation. 

While the level of planning is still out of reach, this summer proved that it was 
necessary to create the state machine program anyway. In this sense, the humans around 
the robot became its planning mechanism. This opened up wide doors for human-robot 
interaction. The end-result of the summer was a demonstration that was shown to 
representatives from DARPA, which was funding a great deal of the research. This demo 
involved a human pointing out the power screwdriver, wheel, and lug nuts to Robonaut, 
and indicating which order Robonaut was supposed to use when tightening the lug nuts. 
Robonaut would confirm its instructions, then reach for the driver, grasp it, then tighten 
each of the lug nuts in order. Of course, because this is real life, sometimes the lug nuts 
wouldn't be tighten all the way, but a set of complex contingencies both in the program 
that tightened the lug nuts and in the state machine allowed Robonaut to detect this error 
and retry several times, until the lug nut was tightened. + 

In addition to the major step of creating the state machine and the work of 
gathering and hand-segmenting the new data that was required for Verbs and Adverbs to 


+ If Robonaut failed several times in a row, it would ask a nearby human for help. 



operate the demo (mainly this data consisted of numerous teleoperator examples of the 
demo described above: grasping a drill and tightening lug nuts), I spend a great deal of 
time on my own research into automatic segmentation methods and on ways of 
incorporating rotations into Verbs and Adverbs. However, the needs of the group 
superseded my own research, and I have not yet succeeded in finding the solutions to 
those very large problems. 


3 Conclusion 

Overall, I would count this year a success, although in unexpected ways. I have 
achieved the goal of acquiring a Master's Degree. I have not found the method of 
segmentation or the method of incorporating rotations into Verbs and Adverbs that will 
make my teaching method truly general, but I have created the next stage of the overall 
intelligent system that will need that teaching method. Research is ongoing, as I am now 
in my second year of the GSRP grant, and intend to apply for a third. 
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